Versions:
mtop is a terminal-based dashboard application published by Elad Sertshuk, designed to serve as an "htop for your local AI" by providing real-time monitoring of local AI servers. Positioned within the system monitoring and developer tools category, the software targets users who run large language models on their own hardware and need visibility into how those models consume resources. It supports several popular local AI server backends, including Ollama, llama.cpp, LM Studio, and vLLM, making it applicable across a range of common self-hosted inference setups rather than being tied to a single platform. The core purpose of mtop is observability and management of local AI workloads directly from the command line. The dashboard displays which models are currently loaded on the server along with the amount of VRAM each one occupies, giving users an at-a-glance understanding of graphics memory utilization. It also surfaces information about the GPU itself, allowing operators to correlate model loading behavior with available hardware capacity. In addition to resource metrics, mtop tracks every request processed by the server together with its throughput expressed in tokens per second, which is useful for evaluating model performance, comparing configurations, and identifying slow or problematic requests during development and testing. Beyond passive monitoring, mtop includes an active management capability: it automatically unloads models that overstay, freeing VRAM when loaded models remain idle longer than they should. This feature addresses a common operational pain point in local AI environments, where memory-constrained GPUs can become saturated by stale model instances that block newer workloads. Typical use cases include developers iterating on applications backed by local inference, researchers benchmarking token generation speeds, and hobbyists managing multiple models on a single machine. The current version of mtop is 1.3.0, and two versions of the software have been released to date, indicating an early but evolving development history under its publisher.
Tags: